Developing an AI application

Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications.

In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application. We'll be using this dataset of 102 flower categories, you can see a few examples below.

The project is broken down into multiple steps:

  • Load and preprocess the image dataset
  • Train the image classifier on your dataset
  • Use the trained classifier to predict image content

We'll lead you through each part which you'll implement in Python.

When you've completed this project, you'll have an application that can be trained on any set of labeled images. Here your network will be learning about flowers and end up as a command line application. But, what you do with your new skills depends on your imagination and effort in building a dataset. For example, imagine an app where you take a picture of a car, it tells you what the make and model is, then looks up information about it. Go build your own dataset and make something new.

First up is importing the packages you'll need. It's good practice to keep all the imports at the beginning of your code. As you work through this notebook and find you need to import a package, make sure to add the import up here.

In [2]:
import signal

from contextlib import contextmanager

import requests


DELAY = INTERVAL = 4 * 60  # interval time in seconds
MIN_DELAY = MIN_INTERVAL = 2 * 60
KEEPALIVE_URL = "https://nebula.udacity.com/api/v1/remote/keep-alive"
TOKEN_URL = "http://metadata.google.internal/computeMetadata/v1/instance/attributes/keep_alive_token"
TOKEN_HEADERS = {"Metadata-Flavor":"Google"}


def _request_handler(headers):
    def _handler(signum, frame):
        requests.request("POST", KEEPALIVE_URL, headers=headers)
    return _handler


@contextmanager
def active_session(delay=DELAY, interval=INTERVAL):
    """
    Example:

    from workspace_utils import active session

    with active_session():
        # do long-running work here
    """
    token = requests.request("GET", TOKEN_URL, headers=TOKEN_HEADERS).text
    headers = {'Authorization': "STAR " + token}
    delay = max(delay, MIN_DELAY)
    interval = max(interval, MIN_INTERVAL)
    original_handler = signal.getsignal(signal.SIGALRM)
    try:
        signal.signal(signal.SIGALRM, _request_handler(headers))
        signal.setitimer(signal.ITIMER_REAL, delay, interval)
        yield
    finally:
        signal.signal(signal.SIGALRM, original_handler)
        signal.setitimer(signal.ITIMER_REAL, 0)


def keep_awake(iterable, delay=DELAY, interval=INTERVAL):
    """
    Example:

    from workspace_utils import keep_awake

    for i in keep_awake(range(5)):
        # do iteration with lots of work here
    """
    with active_session(delay, interval): yield from iterable
In [2]:
# Imports here
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import matplotlib.pyplot as plt
import numpy as np
import torch
from torch import nn
import torch.nn.functional as F
from torch import optim
from torch.optim import lr_scheduler
import torchvision
from torchvision import datasets, transforms, models
import time
import os
import copy
from collections import OrderedDict
import json

Load the data

Here you'll use torchvision to load the data (documentation). The data should be included alongside this notebook, otherwise you can download it here. The dataset is split into three parts, training, validation, and testing. For the training, you'll want to apply transformations such as random scaling, cropping, and flipping. This will help the network generalize leading to better performance. You'll also need to make sure the input data is resized to 224x224 pixels as required by the pre-trained networks.

The validation and testing sets are used to measure the model's performance on data it hasn't seen yet. For this you don't want any scaling or rotation transformations, but you'll need to resize then crop the images to the appropriate size.

The pre-trained networks you'll use were trained on the ImageNet dataset where each color channel was normalized separately. For all three sets you'll need to normalize the means and standard deviations of the images to what the network expects. For the means, it's [0.485, 0.456, 0.406] and for the standard deviations [0.229, 0.224, 0.225], calculated from the ImageNet images. These values will shift each color channel to be centered at 0 and range from -1 to 1.

In [3]:
data_dir = 'flowers'
TRAIN = 'train'
VAL = 'valid'
TEST = 'test'
In [4]:
# TODO: Define your transforms for the training, validation, and testing sets
data_transforms = {
    TRAIN: transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomRotation(30),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    VAL: transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
    TEST: transforms.Compose([
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ]),
}

# TODO: Load the datasets with ImageFolder
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), transform=data_transforms[x])
                  for x in [TRAIN, VAL, TEST]}

# TODO: Using the image datasets and the trainforms, define the dataloaders
dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=32,shuffle=True)
              for x in [TRAIN, VAL, TEST]}
dataset_sizes = {x: len(image_datasets[x]) for x in [TRAIN, VAL, TEST]}
print(dataset_sizes)
class_names = image_datasets[TRAIN].classes
print(class_names)
{'train': 6552, 'valid': 818, 'test': 819}
['1', '10', '100', '101', '102', '11', '12', '13', '14', '15', '16', '17', '18', '19', '2', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '3', '30', '31', '32', '33', '34', '35', '36', '37', '38', '39', '4', '40', '41', '42', '43', '44', '45', '46', '47', '48', '49', '5', '50', '51', '52', '53', '54', '55', '56', '57', '58', '59', '6', '60', '61', '62', '63', '64', '65', '66', '67', '68', '69', '7', '70', '71', '72', '73', '74', '75', '76', '77', '78', '79', '8', '80', '81', '82', '83', '84', '85', '86', '87', '88', '89', '9', '90', '91', '92', '93', '94', '95', '96', '97', '98', '99']

Label mapping

You'll also need to load in a mapping from category label to category name. You can find this in the file cat_to_name.json. It's a JSON object which you can read in with the json module. This will give you a dictionary mapping the integer encoded categories to the actual names of the flowers.

In [5]:
with open('cat_to_name.json', 'r') as f:
    cat_to_name = json.load(f)
    
print(cat_to_name)
{'21': 'fire lily', '3': 'canterbury bells', '45': 'bolero deep blue', '1': 'pink primrose', '34': 'mexican aster', '27': 'prince of wales feathers', '7': 'moon orchid', '16': 'globe-flower', '25': 'grape hyacinth', '26': 'corn poppy', '79': 'toad lily', '39': 'siam tulip', '24': 'red ginger', '67': 'spring crocus', '35': 'alpine sea holly', '32': 'garden phlox', '10': 'globe thistle', '6': 'tiger lily', '93': 'ball moss', '33': 'love in the mist', '9': 'monkshood', '102': 'blackberry lily', '14': 'spear thistle', '19': 'balloon flower', '100': 'blanket flower', '13': 'king protea', '49': 'oxeye daisy', '15': 'yellow iris', '61': 'cautleya spicata', '31': 'carnation', '64': 'silverbush', '68': 'bearded iris', '63': 'black-eyed susan', '69': 'windflower', '62': 'japanese anemone', '20': 'giant white arum lily', '38': 'great masterwort', '4': 'sweet pea', '86': 'tree mallow', '101': 'trumpet creeper', '42': 'daffodil', '22': 'pincushion flower', '2': 'hard-leaved pocket orchid', '54': 'sunflower', '66': 'osteospermum', '70': 'tree poppy', '85': 'desert-rose', '99': 'bromelia', '87': 'magnolia', '5': 'english marigold', '92': 'bee balm', '28': 'stemless gentian', '97': 'mallow', '57': 'gaura', '40': 'lenten rose', '47': 'marigold', '59': 'orange dahlia', '48': 'buttercup', '55': 'pelargonium', '36': 'ruby-lipped cattleya', '91': 'hippeastrum', '29': 'artichoke', '71': 'gazania', '90': 'canna lily', '18': 'peruvian lily', '98': 'mexican petunia', '8': 'bird of paradise', '30': 'sweet william', '17': 'purple coneflower', '52': 'wild pansy', '84': 'columbine', '12': "colt's foot", '11': 'snapdragon', '96': 'camellia', '23': 'fritillary', '50': 'common dandelion', '44': 'poinsettia', '53': 'primula', '72': 'azalea', '65': 'californian poppy', '80': 'anthurium', '76': 'morning glory', '37': 'cape flower', '56': 'bishop of llandaff', '60': 'pink-yellow dahlia', '82': 'clematis', '58': 'geranium', '75': 'thorn apple', '41': 'barbeton daisy', '95': 'bougainvillea', '43': 'sword lily', '83': 'hibiscus', '78': 'lotus lotus', '88': 'cyclamen', '94': 'foxglove', '81': 'frangipani', '74': 'rose', '89': 'watercress', '73': 'water lily', '46': 'wallflower', '77': 'passion flower', '51': 'petunia'}
In [6]:
def imshow(inp, title=None):
    """Imshow for Tensor."""
    inp = inp.numpy().transpose((1, 2, 0))
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    inp = std * inp + mean
    inp = np.clip(inp, 0, 1)
    plt.imshow(inp)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # pause a bit so that plots are updated


# Make a grid from batch
def show_databatch(inputs, labels):
    #print(type(labels))
    classes = torch.add(labels, 1)
    inputs_grid = torchvision.utils.make_grid(inputs)
    imshow(inputs_grid, title=[cat_to_name[str(x.item())] for x in classes])
In [7]:
# Get a batch of training data
inputs, labels = next(iter(dataloaders[TRAIN]))
print(labels)
for i in range(0,29,4):
    show_databatch(inputs[i:i+4], labels[i:i+4])
tensor([ 55,  43,  26,  98,  14,  14,  18,  72,  43,  14,  50,  85,
          6,  90,  49,  75,  74,  54,   6,  64,  77,  23,  66,  69,
         41,  29,  47,  88,  63,  38,  27,  22])

Building and training the classifier

Now that the data is ready, it's time to build and train the classifier. As usual, you should use one of the pretrained models from torchvision.models to get the image features. Build and train a new feed-forward classifier using those features.

We're going to leave this part up to you. Refer to the rubric for guidance on successfully completing this section. Things you'll need to do:

  • Load a pre-trained network (If you need a starting point, the VGG networks work great and are straightforward to use)
  • Define a new, untrained feed-forward network as a classifier, using ReLU activations and dropout
  • Train the classifier layers using backpropagation using the pre-trained network to get the features
  • Track the loss and accuracy on the validation set to determine the best hyperparameters

We've left a cell open for you below, but use as many as you need. Our advice is to break the problem up into smaller parts you can run separately. Check that each part is doing what you expect, then move on to the next. You'll likely find that as you work through each part, you'll need to go back and modify your previous code. This is totally normal!

When training make sure you're updating only the weights of the feed-forward network. You should be able to get the validation accuracy above 70% if you build everything right. Make sure to try different hyperparameters (learning rate, units in the classifier, epochs, etc) to find the best model. Save those hyperparameters to use as default values in the next part of the project.

One last important tip if you're using the workspace to run your code: To avoid having your workspace disconnect during the long-running tasks in this notebook, please read in the earlier page in this lesson called Intro to GPU Workspaces about Keeping Your Session Active. You'll want to include code from the workspace_utils.py module.

Note for Workspace users: If your network is over 1 GB when saved as a checkpoint, there might be issues with saving backups in your workspace. Typically this happens with wide dense layers after the convolutional layers. If your saved checkpoint is larger than 1 GB (you can open a terminal and check with ls -lh), you should reduce the size of your hidden layers and train again.

In [8]:
model = models.vgg16_bn(pretrained=True)
model
Out[8]:
VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
    (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (5): ReLU(inplace)
    (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (7): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (9): ReLU(inplace)
    (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (12): ReLU(inplace)
    (13): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (14): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (16): ReLU(inplace)
    (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (19): ReLU(inplace)
    (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (22): ReLU(inplace)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (26): ReLU(inplace)
    (27): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (28): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (29): ReLU(inplace)
    (30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (31): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (32): ReLU(inplace)
    (33): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (35): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (36): ReLU(inplace)
    (37): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (39): ReLU(inplace)
    (40): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (41): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (42): ReLU(inplace)
    (43): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace)
    (2): Dropout(p=0.5)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace)
    (5): Dropout(p=0.5)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)
In [9]:
# Freeze training for all layers
for param in model.features.parameters():
    param.require_grad = False
In [10]:
num_features = model.classifier[-1].in_features
dropout_p = 0.2
features = list(model.classifier.children())[:-1]
features.extend(nn.Sequential(OrderedDict([
    ('fc1', nn.Linear(num_features, 3000)),
    ('relu1', nn.ReLU(True)),
    ('drop1', nn.Dropout(p=dropout_p)),
    ('fc2', nn.Linear(3000, 2000)),
    ('relu2', nn.ReLU(True)),
    ('drop2', nn.Dropout(p=dropout_p)),
    ('fc3', nn.Linear(2000, 1000)),
    ('relu3', nn.ReLU(True)),
    ('drop3', nn.Dropout(p=dropout_p)),
    ('fc4', nn.Linear(1000, 400)),
    ('relu4', nn.ReLU(True)),
    ('drop4', nn.Dropout(p=dropout_p)),
    ('fc5', nn.Linear(400, len(class_names)))
])))
model.classifier = nn.Sequential(*features)
model
Out[10]:
VGG(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace)
    (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (5): ReLU(inplace)
    (6): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (7): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (8): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (9): ReLU(inplace)
    (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (12): ReLU(inplace)
    (13): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (14): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (15): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (16): ReLU(inplace)
    (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (19): ReLU(inplace)
    (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (22): ReLU(inplace)
    (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (24): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (25): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (26): ReLU(inplace)
    (27): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (28): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (29): ReLU(inplace)
    (30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (31): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (32): ReLU(inplace)
    (33): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (35): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (36): ReLU(inplace)
    (37): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (39): ReLU(inplace)
    (40): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (41): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (42): ReLU(inplace)
    (43): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace)
    (2): Dropout(p=0.5)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace)
    (5): Dropout(p=0.5)
    (6): Linear(in_features=4096, out_features=3000, bias=True)
    (7): ReLU(inplace)
    (8): Dropout(p=0.5)
    (9): Linear(in_features=3000, out_features=2000, bias=True)
    (10): ReLU(inplace)
    (11): Dropout(p=0.5)
    (12): Linear(in_features=2000, out_features=1000, bias=True)
    (13): ReLU(inplace)
    (14): Dropout(p=0.5)
    (15): Linear(in_features=1000, out_features=400, bias=True)
    (16): ReLU(inplace)
    (17): Dropout(p=0.5)
    (18): Linear(in_features=400, out_features=102, bias=True)
  )
)
In [11]:
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print(device)
model = model.to(device)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.classifier.parameters(), lr=0.001)
cuda:0
In [12]:
def eval_model(model, criterion):
    since = time.time()
    avg_loss = 0
    avg_acc = 0
    loss_test = 0
    acc_test = 0
    
    test_batches = len(dataloaders[TEST]) # 26 batches
    print("Evaluating model")
    print('-' * 10)
    
    with torch.no_grad():
        for i, data in enumerate(dataloaders[TEST]): # test batch has 32 images
            if i % 10 == 0:
                print("\nTest batch {}/{}".format(i+1, test_batches))

            model.train(False)
            model.eval()
            inputs, labels = data

        
            inputs, labels = inputs.to(device), labels.to(device) # inputs shape (32,3,224,224) labels shape (32x1)
            print("\nLabels: {}".format(labels))

            outputs = model.forward(inputs) # outputs shape (32,102)
            #print("\nOutputs Shape: {}",format(outputs.shape))
            #print("\nOutputs[0]: {}".format(outputs[0]))
            _, preds = torch.max(outputs.data, 1) # preds are indices of each max output (32x1)
            #print("\n_: {}".format(_))
            #print("\nPredictions Shape: {}".format(preds.shape))
            print("\nPredic: {}".format(preds))
            loss = criterion(outputs, labels)
            
            loss_test += loss.item()
            acc_test += torch.sum(preds == labels.data).item() # This compares how many preds equal to label
            
            del inputs, labels, outputs, preds
            torch.cuda.empty_cache()
        
        avg_loss = loss_test / dataset_sizes[TEST]
        avg_acc = acc_test / dataset_sizes[TEST]
        
        elapsed_time = time.time() - since
        print()
        print("Evaluation completed in {:.0f}m {:.0f}s".format(elapsed_time // 60, elapsed_time % 60))
        print("Avg loss (test): {:.4f}".format(avg_loss))
        print("Avg acc (test): {:.4f}".format(avg_acc))
        print('-' * 10)

with active_session():
    eval_model(model, criterion)
Evaluating model
----------

Test batch 1/26

Labels: tensor([ 91,  66,   7,   9,  71,  17,  46,  96,  77,  10,   5,  33,
         32,  77,  71,  26,  13,  43,  20,   2,   7,  82,  40,  61,
         49,  90,  96,  72,  82,   5,  93,  88], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  90,  46,  46,
         46,  46,  90,  46,  46,  46,  76,  46,  46,  46,  25,  46,
         76,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 11,  60,  49,  26,  92,  11,  94,  77,  40,  79,  83,  33,
         78,  57,  51,   5,  44,  51,  59,  82,  36,  31,  80,  30,
         73,  89,  72,  73,  12,  55,   6,  15], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  76,  46,  46,  25,  46,  46,  46,  46,  76,  46,  46,
         46,  46,  46,  25,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([  77,   83,   54,    7,   46,   83,   80,   67,   86,   17,
         101,    0,   57,   67,   95,   44,   60,   35,   60,   26,
          12,    6,   89,   57,   62,   34,   71,   41,   87,   47,
          12,   72], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  90,  46,  76,  46,  46,  46,  46,  46,
         46,  46,  46,  76,  46,  46,  46,  46,  76,  46,  46,  46,
         90,  90,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 45,  85,  24,  73,  10,  24,  12,  78,  55,  13,  19,  83,
         54,  85,  73,  11,  51,  40,  98,  51,  48,  81,  43,  29,
         75,  73,  20,  78,  73,  73,  50,   6], device='cuda:0')

Predic: tensor([ 83,  46,  46,  46,  76,  46,  46,  46,  83,  46,  46,  46,
         46,  46,  46,  46,  90,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  76,  46,  46,  46], device='cuda:0')

Labels: tensor([ 98,  60,  83,  92,  96,  45,  89,  96,  47,  58,  94,  40,
         40,  93,  94,  89,  64,  92,  56,  82,  36,  53,  33,   6,
         37,  34,  21,  47,  30,  80,  83,  92], device='cuda:0')

Predic: tensor([ 90,  46,  46,  46,  46,  76,  78,  46,  46,  90,  46,  46,
         25,  46,  46,  46,  46,  25,  46,  46,  46,  46,  46,  76,
         46,  46,  76,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 72,  73,  33,  85,  34,  18,  43,   2,  86,  93,   5,   5,
         96,  77,  38,  71,  92,  33,   6,  43,  22,  41,  88,  95,
         90,  58,  86,  96,  49,  74,  96,  84], device='cuda:0')

Predic: tensor([ 46,  90,  46,  46,  46,  46,  46,  46,  46,  46,  76,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  76,  46,  46], device='cuda:0')

Labels: tensor([ 79,  40,  54,  43,  99,  49,  31,  29,   4,  75,  19,  92,
         58,  32,  51,  73,  90,  48,  78,  90,  56,  98,  72,  84,
         74,  96,  23,  41,  26,  43,  37,  41], device='cuda:0')

Predic: tensor([ 46,  46,  46,  90,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  25,  90,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 40,  77,  23,  83,  73,   8,  50,  20,  72,  75,  88,  55,
         80,  76,  75,  11,  81,  80,  83,  85,  55,  96,  25,  82,
         14,  43,  42,  26,  89,  23,   5,  90], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  90,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  25,  46,
         46,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([  29,   39,   50,   89,   77,    2,   74,    1,   59,    4,
          73,   28,   95,   29,  101,   83,   55,   95,   94,   40,
          78,   87,   54,   78,    6,   73,   96,   50,   71,   13,
          99,   75], device='cuda:0')

Predic: tensor([ 46,  76,  46,  46,  46,  46,  76,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         25,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 40,  91,  21,  84,  99,  20,  31,  54,  28,  34,  96,   3,
         43,  60,  66,  74,  28,  49,  49,  72,  94,  36,  30,  65,
         98,   8,  34,  94,  77,  86,  68,  43], device='cuda:0')

Predic: tensor([ 25,  46,  46,  46,  46,  25,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  76,  46,  46,  25,  46,  46,  90], device='cuda:0')

Test batch 11/26

Labels: tensor([  74,   73,   39,   69,   14,    9,    3,   92,   90,   49,
          22,   43,   71,   78,   97,   96,   56,   56,   28,   82,
           2,   59,   77,   37,   11,  101,   84,   83,   75,   81,
          80,   51], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  25,  83,  46,  76,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 82,  75,   4,  32,  56,  57,   5,  96,  90,  64,  71,  64,
         34,  38,  61,  81,  65,  72,  73,  40,  90,  53,  20,  99,
          2,  32,  70,  73,  90,  16,  56,  87], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  90,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  25,  46,  46,  46], device='cuda:0')

Labels: tensor([  24,   52,   74,   39,   75,   41,   77,   62,   89,   51,
          88,  101,   51,   40,    9,   74,   43,   26,   94,   48,
          15,   71,   73,   49,   55,   81,    6,   90,   64,   75,
          43,   63], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  76,  46,  46,  76,  46,  46,  46,  25,
         46,  46,  46,  46,  90,  90,  46,  46,  46,  46,  46,  46,
         46,  46,  76,  46,  76,  46,  46,  46], device='cuda:0')

Labels: tensor([   4,   73,   80,   48,   95,   56,   32,   93,   60,   72,
          94,   43,   97,   43,   37,   94,  100,   84,   82,    1,
         101,   93,   40,   58,   24,    4,   53,  100,   21,   60,
          62,   89], device='cuda:0')

Predic: tensor([ 46,  90,  83,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  90,  46,  46,  46,  46,  90,  46,  46,  90,  46,  76,
         46,  46,  46,  46,  78,  46,  76,  46], device='cuda:0')

Labels: tensor([  86,   66,   84,   73,   84,   38,   69,   30,   74,  101,
          78,   33,   43,   73,   11,   56,   55,   57,   83,   58,
          50,   23,   49,   87,   81,   97,   90,   57,   41,   24,
          68,   40], device='cuda:0')

Predic: tensor([ 46,  46,  90,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  90,  46,  46,  46,  78,  46,  90,  46,  46,  46,  46,
         46,  46,  46,  76,  46,  46,  90,  46], device='cuda:0')

Labels: tensor([  2,  77,  82,  11,  83,  18,  63,  72,  77,  85,  93,  65,
         37,  77,  13,  27,  49,  98,  33,  24,  74,  85,  93,  49,
          0,  29,  59,   4,  56,  40,  82,  97], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  76,  46,
         46,  46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 18,  64,  97,  27,  49,  89,  17,  15,  58,  31,  34,  78,
         84,  77,  73,  82,  86,  75,  56,  58,  49,  41,  67,   0,
         39,  73,  43,  45,   3,  87,  53,   7], device='cuda:0')

Predic: tensor([ 46,  76,  46,  46,  46,  46,  76,  46,  25,  46,  46,  46,
         46,  46,  46,  46,  46,  83,  46,  76,  46,  46,  46,  46,
         76,  46,  90,  76,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 42,  49,  97,  10,  49,  66,  77,  48,  54,  86,  45,  98,
         81,  49,  55,  38,  97,  49,  97,  59,  16,  26,  21,  24,
         29,  85,  61,  18,  83,  33,  49,  12], device='cuda:0')

Predic: tensor([ 46,  46,  46,  25,  46,  46,  46,  46,  46,  46,  76,  46,
         46,  46,  46,  46,  46,  46,  78,  46,  46,  46,  46,  76,
         46,  46,  46,  46,  46,  46,  46,  25], device='cuda:0')

Labels: tensor([ 26,  23,  70,  46,  53,  38,  59,  78,  28,  99,  80,  45,
         86,  88,  92,  92,  58,  90,  43,  41,  92,  26,  78,  38,
         76,  38,  38,  85,  97,  50,  26,  79], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  25,  46,  46,  46,  46,  46,  83,  78,  46,  46,
         46,  46,  46,  46,  83,  46,  25,  46], device='cuda:0')

Labels: tensor([  2,  59,  29,  31,  73,  51,  49,  48,  76,  97,  62,  21,
         69,  43,  25,  32,  72,  56,  38,  36,   3,  74,  97,  43,
         89,  51,  29,  74,  13,  73,  81,   7], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  76,  46,
         25,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  25,  46,  46,  46,  46], device='cuda:0')

Test batch 21/26

Labels: tensor([  14,   52,   77,   36,   73,   70,   54,   97,   23,  100,
          17,   89,    0,   38,   69,   53,   26,   55,   49,   35,
          38,   49,   98,   71,   14,    8,   26,   78,   51,   41,
          80,   77], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  90,  78,  90,  46,  46,  78,
         46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  90,  46,  25], device='cuda:0')

Labels: tensor([ 58,  18,  78,  75,  48,  77,  63,  18,  36,  55,  56,  30,
         41,  81,  49,  77,  35,  37,  11,  35,  94,  37,  54,  57,
         86,  69,  93,  92,  41,  11,  77,  85], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46,  46,  90,  46,  46,  90,
         46,  46,  46,  46,  46,  46,  90,  46], device='cuda:0')

Labels: tensor([  77,   51,   77,   76,  100,   73,   59,   73,    5,   83,
           6,   77,   82,   49,   64,   77,   60,   82,   40,   32,
          28,   73,   96,   59,   48,   24,   14,   43,   34,   50,
          56,   81], device='cuda:0')

Predic: tensor([ 46,  90,  46,  46,  46,  46,  76,  46,  46,  46,  25,  46,
         46,  46,  90,  46,  46,  46,  46,  46,  90,  46,  46,  46,
         46,  46,  46,  83,  46,  46,  46,  90], device='cuda:0')

Labels: tensor([  47,   74,   52,   51,   75,   53,   38,   88,   90,   97,
          38,   26,   69,   50,   18,   84,   89,   74,   78,   97,
           1,   64,   80,   31,  101,   84,   96,   83,   86,   92,
          75,    2], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  25,  46,  46,  83,  46,  46,  46,  46,
         25,  46,  46,  46,  46,  46,  46,  46,  46,  76,  46,  46,
         46,  46,  46,  46,  46,  46,  25,  46], device='cuda:0')

Labels: tensor([ 24,  95,  90,  70,  90,  98,  68,  26,  94,  65,   6,  13,
         84,  51,   9,  84,  12,  43,   7,  55,  35,  13,  37,  44,
         49,   5,  84,  63,   0,  40,  92,  61], device='cuda:0')

Predic: tensor([ 46,  46,  90,  25,  46,  46,  90,  46,  46,  46,  46,  46,
         46,  90,  76,  46,  46,  90,  90,  46,  46,  46,  46,  76,
         46,  46,  46,  90,  46,  46,  46,  46], device='cuda:0')

Labels: tensor([ 96,  92,  85,  83,  42,  83,  56,  59,  89,  98,  74,  81,
         84,  73,  22,  52,  77,  63,  38], device='cuda:0')

Predic: tensor([ 46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,  46,
         46,  46,  46,  46,  46,  46,  46], device='cuda:0')

Evaluation completed in 0m 26s
Avg loss (test): 0.1468
Avg acc (test): 0.0049
----------
In [13]:
def visualize_model(model, num_images=16):
    was_training = model.training
    
    # Set model for evaluation
    model.train(False)
    model.eval() 
    
    images_so_far = 0

    with torch.no_grad():
        for i, data in enumerate(dataloaders[TEST]):
            inputs, labels = data
            size = inputs.shape[0]
            print(size)
            inputs, labels = inputs.to(device), labels.to(device)
        
            outputs = model.forward(inputs)
        
            _, preds = torch.max(outputs.data, 1)
            predicted_labels = torch.stack([preds[j] for j in range(size)])
        
            for j in range(0,num_images,4):
                print("Ground truth:")
                show_databatch(inputs[j:j+4].data.cpu(), labels[j:j+4].data.cpu())
                print("Prediction:")
                show_databatch(inputs[j:j+4].data.cpu(), predicted_labels[j:j+4])
        
            del inputs, labels, outputs, preds, predicted_labels
            torch.cuda.empty_cache()
        
            images_so_far += size
            if images_so_far >= num_images:
                break
        
    model.train(was_training) # Revert model back to original training state
    
with active_session():
    visualize_model(model)
32
Ground truth:
Prediction:
Ground truth:
Prediction:
Ground truth:
Prediction:
Ground truth:
Prediction:
In [16]:
# TODO: Build and train your network
def train_model(model, criterion, optimizer, num_epochs=25):
    since = time.time()
    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0
    
    avg_loss = 0
    avg_acc = 0
    avg_loss_val = 0
    avg_acc_val = 0
    
    train_batches = len(dataloaders[TRAIN])
    val_batches = len(dataloaders[VAL])
    
    for epoch in range(num_epochs):
        print("Epoch {}/{}".format(epoch+1, num_epochs))
        print('-' * 10)
        
        loss_train = 0
        loss_val = 0
        acc_train = 0
        acc_val = 0
        
        model.train(True)
        
        for i, data in enumerate(dataloaders[TRAIN]):
            if i % 100 == 0:
                print("\nTraining batch {}/{}".format(i+1, train_batches))
                
            inputs, labels = data
            
            inputs, labels = inputs.to(device), labels.to(device)
            
            optimizer.zero_grad()
            
            outputs = model.forward(inputs)
            print("\rFeed forward for training batch {} is done.".format(i+1), end='', flush=True)
            
            _, preds = torch.max(outputs.data, 1)
            loss = criterion(outputs, labels)
            print("\rCalculating loss for training batch {} is done.".format(i+1), end='', flush=True)
            
            loss.backward()
            optimizer.step()
            print("\rBack propagation for training batch {} is done.".format(i+1), end='', flush=True)
            
            loss_train += loss.item()
            acc_train += torch.sum(preds == labels.data).item()
            
            del inputs, labels, outputs, preds
            torch.cuda.empty_cache()
        
        print()
        avg_loss = loss_train / dataset_sizes[TRAIN]
        avg_acc = acc_train / dataset_sizes[TRAIN]
        
        model.train(False)
        model.eval()
        
        with torch.no_grad():
            for i, data in enumerate(dataloaders[VAL]):
                if i % 10 == 0:
                    print("\nValidation batch {}/{}".format(i+1, val_batches))
                
                inputs, labels = data
            
                inputs, labels = inputs.to(device), labels.to(device)
            
                optimizer.zero_grad()
            
                outputs = model.forward(inputs)
                print("\rFeed forward for validation batch {} is done.".format(i+1), end='', flush=True)
            
                _, preds = torch.max(outputs.data, 1)
                loss = criterion(outputs, labels)
                print("\rCalculating loss for validation batch {} is done.".format(i+1), end='', flush=True)
            
                loss_val += loss.item()
                acc_val += torch.sum(preds == labels.data).item()
            
                del inputs, labels, outputs, preds
                torch.cuda.empty_cache()
        
            avg_loss_val = loss_val / dataset_sizes[VAL]
            avg_acc_val = acc_val / dataset_sizes[VAL]
        
        print()
        print("\nEpoch {} result: ".format(epoch+1))
        print("Avg loss (train): {:.4f}".format(avg_loss))
        print("Avg acc (train): {:.4f}".format(avg_acc))
        print("Avg loss (val): {:.4f}".format(avg_loss_val))
        print("Avg acc (val): {:.4f}".format(avg_acc_val))
        print('-' * 10)
        print()
        
        if avg_acc_val > best_acc:
            best_acc = avg_acc_val
            best_model_wts = copy.deepcopy(model.state_dict())
        
    elapsed_time = time.time() - since
    print()
    print("Training completed in {:.0f}m {:.0f}s".format(elapsed_time // 60, elapsed_time % 60))
    print("Best acc: {:.4f}".format(best_acc))
    
    model.load_state_dict(best_model_wts)
    return model
In [17]:
with active_session():
    model = train_model(model, criterion, optimizer, num_epochs=25)
Epoch 1/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 1 result: 
Avg loss (train): 0.1416
Avg acc (train): 0.0302
Avg loss (val): 0.1414
Avg acc (val): 0.0452
----------

Epoch 2/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 2 result: 
Avg loss (train): 0.1381
Avg acc (train): 0.0446
Avg loss (val): 0.1325
Avg acc (val): 0.0611
----------

Epoch 3/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 3 result: 
Avg loss (train): 0.1335
Avg acc (train): 0.0583
Avg loss (val): 0.1287
Avg acc (val): 0.0660
----------

Epoch 4/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 4 result: 
Avg loss (train): 0.1318
Avg acc (train): 0.0580
Avg loss (val): 0.1275
Avg acc (val): 0.0819
----------

Epoch 5/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 5 result: 
Avg loss (train): 0.1297
Avg acc (train): 0.0604
Avg loss (val): 0.1280
Avg acc (val): 0.0807
----------

Epoch 6/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 6 result: 
Avg loss (train): 0.1284
Avg acc (train): 0.0705
Avg loss (val): 0.1204
Avg acc (val): 0.0990
----------

Epoch 7/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 7 result: 
Avg loss (train): 0.1277
Avg acc (train): 0.0728
Avg loss (val): 0.1292
Avg acc (val): 0.0966
----------

Epoch 8/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 8 result: 
Avg loss (train): 0.1249
Avg acc (train): 0.0850
Avg loss (val): 0.1182
Avg acc (val): 0.1112
----------

Epoch 9/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 9 result: 
Avg loss (train): 0.1258
Avg acc (train): 0.0853
Avg loss (val): 0.1202
Avg acc (val): 0.1247
----------

Epoch 10/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 10 result: 
Avg loss (train): 0.1257
Avg acc (train): 0.0809
Avg loss (val): 0.1183
Avg acc (val): 0.1161
----------

Epoch 11/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 11 result: 
Avg loss (train): 0.1236
Avg acc (train): 0.0897
Avg loss (val): 0.1163
Avg acc (val): 0.1125
----------

Epoch 12/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 12 result: 
Avg loss (train): 0.1238
Avg acc (train): 0.0914
Avg loss (val): 0.1152
Avg acc (val): 0.1186
----------

Epoch 13/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 13 result: 
Avg loss (train): 0.1242
Avg acc (train): 0.0899
Avg loss (val): 0.1149
Avg acc (val): 0.1100
----------

Epoch 14/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 14 result: 
Avg loss (train): 0.1220
Avg acc (train): 0.0936
Avg loss (val): 0.1142
Avg acc (val): 0.1308
----------

Epoch 15/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 15 result: 
Avg loss (train): 0.1227
Avg acc (train): 0.0929
Avg loss (val): 0.1167
Avg acc (val): 0.1357
----------

Epoch 16/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 16 result: 
Avg loss (train): 0.1226
Avg acc (train): 0.0981
Avg loss (val): 0.1161
Avg acc (val): 0.1357
----------

Epoch 17/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 17 result: 
Avg loss (train): 0.1222
Avg acc (train): 0.0972
Avg loss (val): 0.1126
Avg acc (val): 0.1491
----------

Epoch 18/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 18 result: 
Avg loss (train): 0.1213
Avg acc (train): 0.1013
Avg loss (val): 0.1128
Avg acc (val): 0.1467
----------

Epoch 19/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 19 result: 
Avg loss (train): 0.1236
Avg acc (train): 0.0992
Avg loss (val): 0.1126
Avg acc (val): 0.1430
----------

Epoch 20/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 20 result: 
Avg loss (train): 0.1239
Avg acc (train): 0.0920
Avg loss (val): 0.1135
Avg acc (val): 0.1357
----------

Epoch 21/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 21 result: 
Avg loss (train): 0.1223
Avg acc (train): 0.0991
Avg loss (val): 0.1125
Avg acc (val): 0.1516
----------

Epoch 22/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 22 result: 
Avg loss (train): 0.1212
Avg acc (train): 0.1030
Avg loss (val): 0.1109
Avg acc (val): 0.1614
----------

Epoch 23/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 23 result: 
Avg loss (train): 0.1239
Avg acc (train): 0.0978
Avg loss (val): 0.1166
Avg acc (val): 0.1467
----------

Epoch 24/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 24 result: 
Avg loss (train): 0.1214
Avg acc (train): 0.1023
Avg loss (val): 0.1102
Avg acc (val): 0.1406
----------

Epoch 25/25
----------

Training batch 1/205
Back propagation for training batch 100 is done.
Training batch 101/205
Back propagation for training batch 200 is done.
Training batch 201/205
Back propagation for training batch 205 is done.

Validation batch 1/26
Calculating loss for validation batch 10 is done.
Validation batch 11/26
Calculating loss for validation batch 20 is done.
Validation batch 21/26
Calculating loss for validation batch 26 is done.
Epoch 25 result: 
Avg loss (train): 0.1221
Avg acc (train): 0.1053
Avg loss (val): 0.1094
Avg acc (val): 0.1467
----------


Training completed in 221m 10s
Best acc: 0.1614

Testing your network

It's good practice to test your trained network on test data, images the network has never seen either in training or validation. This will give you a good estimate for the model's performance on completely new images. Run the test images through the network and measure the accuracy, the same way you did validation. You should be able to reach around 70% accuracy on the test set if the model has been trained well.

In [1]:
# TODO: Do validation on the test set
with active_session():
    eval_model(model, criterion)
    visualize_model(model)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-e60107b08fe3> in <module>()
      1 # TODO: Do validation on the test set
----> 2 with active_session():
      3     eval_model(model, criterion)
      4     visualize_model(model)

NameError: name 'active_session' is not defined

Save the checkpoint

Now that your network is trained, save the model so you can load it later for making predictions. You probably want to save other things such as the mapping of classes to indices which you get from one of the image datasets: image_datasets['train'].class_to_idx. You can attach this to the model as an attribute which makes inference easier later on.

model.class_to_idx = image_datasets['train'].class_to_idx

Remember that you'll want to completely rebuild the model later so you can use it for inference. Make sure to include any information you need in the checkpoint. If you want to load the model and keep training, you'll want to save the number of epochs as well as the optimizer state, optimizer.state_dict. You'll likely want to use this trained model in the next part of the project, so best to save it now.

In [ ]:
# TODO: Save the checkpoint
t = time.localtime()
timestamp = time.strftime('%b-%d-%Y_%H%M', t)
checkpoint = {
    'model_state_dict': model.state_dict(),
    'optimizer_state_dict': optimizer.state_dict()}

torch.save(checkpoint, 'checkpoint_{}.pth'.format(timestamp))

Loading the checkpoint

At this point it's good to write a function that can load a checkpoint and rebuild the model. That way you can come back to this project and keep working on it without having to retrain the network.

In [ ]:
# TODO: Write a function that loads a checkpoint and rebuilds the model

Inference for classification

Now you'll write a function to use a trained network for inference. That is, you'll pass an image into the network and predict the class of the flower in the image. Write a function called predict that takes an image and a model, then returns the top $K$ most likely classes along with the probabilities. It should look like

probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']

First you'll need to handle processing the input image such that it can be used in your network.

Image Preprocessing

You'll want to use PIL to load the image (documentation). It's best to write a function that preprocesses the image so it can be used as input for the model. This function should process the images in the same manner used for training.

First, resize the images where the shortest side is 256 pixels, keeping the aspect ratio. This can be done with the thumbnail or resize methods. Then you'll need to crop out the center 224x224 portion of the image.

Color channels of images are typically encoded as integers 0-255, but the model expected floats 0-1. You'll need to convert the values. It's easiest with a Numpy array, which you can get from a PIL image like so np_image = np.array(pil_image).

As before, the network expects the images to be normalized in a specific way. For the means, it's [0.485, 0.456, 0.406] and for the standard deviations [0.229, 0.224, 0.225]. You'll want to subtract the means from each color channel, then divide by the standard deviation.

And finally, PyTorch expects the color channel to be the first dimension but it's the third dimension in the PIL image and Numpy array. You can reorder dimensions using ndarray.transpose. The color channel needs to be first and retain the order of the other two dimensions.

In [ ]:
def process_image(image):
    ''' Scales, crops, and normalizes a PIL image for a PyTorch model,
        returns an Numpy array
    '''
    
    # TODO: Process a PIL image for use in a PyTorch model

To check your work, the function below converts a PyTorch tensor and displays it in the notebook. If your process_image function works, running the output through this function should return the original image (except for the cropped out portions).

In [ ]:
def imshow(image, ax=None, title=None):
    """Imshow for Tensor."""
    if ax is None:
        fig, ax = plt.subplots()
    
    # PyTorch tensors assume the color channel is the first dimension
    # but matplotlib assumes is the third dimension
    image = image.numpy().transpose((1, 2, 0))
    
    # Undo preprocessing
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    image = std * image + mean
    
    # Image needs to be clipped between 0 and 1 or it looks like noise when displayed
    image = np.clip(image, 0, 1)
    
    ax.imshow(image)
    
    return ax

Class Prediction

Once you can get images in the correct format, it's time to write a function for making predictions with your model. A common practice is to predict the top 5 or so (usually called top-$K$) most probable classes. You'll want to calculate the class probabilities then find the $K$ largest values.

To get the top $K$ largest values in a tensor use x.topk(k). This method returns both the highest k probabilities and the indices of those probabilities corresponding to the classes. You need to convert from these indices to the actual class labels using class_to_idx which hopefully you added to the model or from an ImageFolder you used to load the data (see here). Make sure to invert the dictionary so you get a mapping from index to class as well.

Again, this method should take a path to an image and a model checkpoint, then return the probabilities and classes.

probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']
In [ ]:
def predict(image_path, model, topk=5):
    ''' Predict the class (or classes) of an image using a trained deep learning model.
    '''
    
    # TODO: Implement the code to predict the class from an image file

Sanity Checking

Now that you can use a trained model for predictions, check to make sure it makes sense. Even if the testing accuracy is high, it's always good to check that there aren't obvious bugs. Use matplotlib to plot the probabilities for the top 5 classes as a bar graph, along with the input image. It should look like this:

You can convert from the class integer encoding to actual flower names with the cat_to_name.json file (should have been loaded earlier in the notebook). To show a PyTorch tensor as an image, use the imshow function defined above.

In [ ]:
# TODO: Display an image along with the top 5 classes